[Go] Add Go bindings for ONNX Runtime C API by dannyota · Pull Request #29615 · microsoft/onnxruntime

dannyota · 2026-07-08T06:05:04Z

Description

Official Go bindings for the ONNX Runtime C API via CGO, following
the pattern of existing language bindings (rust/, java/, csharp/).

Closes #9786

Motivation

Go is widely used for backend services that need ML inference
(embeddings, classification, NLP). There is strong community demand
(#9786, 19 comments) and no official Go support. This PR fills that gap.

What's included

Session management: create from file or byte buffer, introspect
inputs/outputs with dynamic dimension support
Tensor I/O: generic CreateTensor[T] (Go 1.18+ generics),
untyped NewTensorFromBytes, string tensors, zero-length dimensions
Session options: thread counts, graph optimization, execution mode,
memory arena, profiling, free dimension overrides
Execution providers: CUDA (V2), TensorRT (V2), generic string
key-value API for all other EPs
IO Binding: bind inputs/outputs to devices for GPU workflows
Model metadata: producer, graph name, domain, version, custom keys
Run options: log level, tag, config entries
Cancellation: context.Context support via RunOptionsSetTerminate
Concurrency: Session.Run safe for concurrent use (ORT guarantee)
Sequence/Map: value type inspection and sequence element access

Design

Wraps the C API vtable (OrtApi) through ~80 C shim functions in
cshim.h/cshim.c — CGO cannot call C function pointers directly.
Library loaded at runtime via dlopen (Linux/macOS) or LoadDLL
(Windows). Singleton OrtEnv per process via sync.Mutex.

Zero-copy tensor inputs via runtime.Pinner. Output tensors wrap
ORT-allocated memory with views valid until Close().

Module path: github.com/microsoft/onnxruntime/go
Package: onnxruntime (import as ort)
Go 1.26 minimum. stdlib + CGO only — no third-party dependencies.

Tests

46 tests passing with -race:

Lifecycle, inference correctness, dynamic shapes, zero-length dims
All tensor types (float32, int64, bool, string, bytes, scalars)
Error handling, type/shape mismatch, use-after-close
Concurrency: 16 goroutines × 50 runs on one session
Context cancellation, IO binding, model metadata, run options
Integration: Qwen3-Embedding-0.6B ONNX INT8 (59 inputs, 57 outputs)
Semgrep Pro: 0 findings (195 rules)
golangci-lint: 0 issues

Idiomatic Go wrapper for the ORT C API via CGO, matching the pattern of existing language bindings (rust/, java/, csharp/). Covers session management, typed/untyped tensor I/O, string tensors, IO binding, model metadata, run options, session options with CUDA and TensorRT execution providers, context cancellation, and concurrent inference. Tested with real Qwen3-Embedding-0.6B ONNX INT8 model.

dannyota · 2026-07-08T06:09:01Z

@microsoft-github-policy-service agree

Fix TOCTOU race in NewIOBinding (use-after-free on concurrent Close), empty outputNames panic in Run/RunWithOptions, and metadata use-after- close segfault. Add 15 tests covering all identified coverage gaps. Remove dead code. Bump minimum ORT version to 1.27.0 for getter APIs.

dannyota · 2026-07-08T07:20:22Z

Updated with a second commit:

Fixed TOCTOU race in NewIOBinding (use-after-free on concurrent Close)
Fixed empty outputNames panic in Run/RunWithOptions
Fixed metadata use-after-close segfault
Added 15 tests (63 total), covering all tensor types, error paths, IO binding, and edge cases
Bumped minimum ORT to 1.27.0 for GetMemPatternEnabled/GetSessionExecutionMode getters
Tested against ORT 1.28.0 built from rel-1.28.0 branch — all passing

Try API version 27 first, fall back to 17 for ORT 1.17-1.26. GetExecutionMode and IsMemPatternEnabled return a clear error on older libraries instead of crashing. Minimum ORT is now 1.17.

dannyota · 2026-07-08T09:39:17Z

Added API version fallback: bindings now try API version 27 first, fall back to 17 for older ORT libraries. This means ORT 1.17+ is supported (previously required 1.27+). The two getter functions added in 1.27 (GetExecutionMode, IsMemPatternEnabled) return a clear error on older libraries instead of crashing.

Add GetVersion/APIVersion, cache CString names in Session for zero- alloc Run hot path, enable/disable telemetry, SetOptimizedModelFilePath, RegisterCustomOpsLibrary, Has/GetSessionConfigEntry, NewSequence, NewMap, NewMapFromGoMap. Fix API version probing to try 27 down to 17 for broader ORT compatibility. Fix IsSequence/IsMap enum values.

Copilot

Pull request overview

This PR introduces an official Go (cgo) binding layer for the ONNX Runtime C API, including runtime library loading, core session/tensor APIs, and a fairly comprehensive Go test suite with small ONNX model fixtures.

Changes:

Added Go package go/onnxruntime with wrappers for environment lifecycle, sessions, tensors (typed + raw-bytes + string), session/run options, IO binding, and model metadata.
Added a C shim layer (cshim.h/cshim.c) to call through the OrtApi vtable (since cgo can’t call function pointers directly).
Added test models and Go unit/integration tests covering common workflows (including dynamic shapes, zero-length dims, concurrency, and optional Qwen3 integration).

Reviewed changes

Copilot reviewed 29 out of 34 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
go/testdata/gen_models.py	Generates minimal ONNX models used by the Go tests.
go/onnxruntime/types.go	Defines Go enums/types for tensor dtypes, graph optimization level, execution mode, and generic tensor element constraints.
go/onnxruntime/tensor.go	Implements typed tensors, raw-byte tensors, sequence/map wrappers, and tensor shape/type helpers.
go/onnxruntime/tensor_test.go	Unit tests for tensor creation, accessors, and sequence/map helpers.
go/onnxruntime/string_tensor.go	Implements string tensor creation and string extraction.
go/onnxruntime/string_tensor_test.go	Unit tests for string tensor creation and reading.
go/onnxruntime/session.go	Implements session creation, IO introspection, Run/RunWithOptions, profiling end, and lifecycle management.
go/onnxruntime/session_test.go	Session tests covering model IO, inference, dynamic/zero-len dims, concurrency, cancellation, and options.
go/onnxruntime/run_options.go	Run options wrapper (verbosity, severity, tag, terminate, config entries).
go/onnxruntime/run_options_test.go	Tests for run options and RunWithOptions behavior.
go/onnxruntime/qwen3_test.go	Optional integration test for a locally available Qwen3 embedding model.
go/onnxruntime/ort.go	Global initialization/shutdown, telemetry toggles, provider enumeration, and library path resolution.
go/onnxruntime/ort_test.go	Package TestMain + tests for init/idempotency, providers, telemetry, shutdown constraints, and OrtError typing.
go/onnxruntime/options.go	SessionOptions wrapper including EP configuration, profiling, free-dim overrides, getters (API-gated), and misc config.
go/onnxruntime/options_test.go	Tests for SessionOptions cloning, memory settings, execution mode, getters, profiling, EP errors, and config entries.
go/onnxruntime/onnxruntime_error_code.h	Error-code header copy for the shim/include surface.
go/onnxruntime/metadata.go	Model metadata wrapper (producer, graph, domain, description, version, custom KV).
go/onnxruntime/metadata_test.go	Tests for metadata access, double-close, and use-after-close behavior.
go/onnxruntime/load_windows.go	Windows dynamic loading via `LoadDLL` + `FindProc`.
go/onnxruntime/load_unix.go	Unix dynamic loading via `dlopen`/`dlsym`.
go/onnxruntime/iobinding.go	IO binding API (bind inputs/outputs, run, fetch bound outputs/names).
go/onnxruntime/iobinding_test.go	IO binding tests (basic run, output binding modes, output names/values, memory info, closed session).
go/onnxruntime/errors.go	Go error mapping for OrtStatus + wrapping helpers.
go/onnxruntime/doc.go	Package-level documentation for initialization and concurrency expectations.
go/onnxruntime/cshim.h	Declares the C shim surface used by cgo.
go/onnxruntime/cshim.c	Implements shim functions that forward to `OrtApi` function pointers.
go/go.mod	Declares Go module path and minimum Go version.
go/.golangci.yml	Lint configuration for the Go code.
go/.gitignore	Ignores local `.ort-lib/` test library directory.

+func checkInit() error {
+	if !initialized {
+		if shutdown {
+			return errShutdown
+		}
+		return errNotInitialized
+	}
+	return nil
+}


+	count := shapeElementCount(t.shape)
+	if count == 0 {
+		return nil, nil
+	}
+
+	var dataPtr unsafe.Pointer
+	if err := checkStatus(C.ort_GetTensorMutableData(t.value, &dataPtr)); err != nil {
+		return nil, wrapErr("tensor data", err)
+	}
+
+	return unsafe.Slice((*T)(dataPtr), count), nil
+}


+	count := shapeElementCount(t.shape)
+	if count == 0 {
+		return nil, nil
+	}
+
+	var dataPtr unsafe.Pointer
+	if err := checkStatus(C.ort_GetTensorMutableData(t.value, &dataPtr)); err != nil {
+		return nil, wrapErr("tensor bytes", err)
+	}
+
+	nbytes := int(count) * elemSize(t.dtype)
+	return unsafe.Slice((*byte)(dataPtr), nbytes), nil
+}


+func shapeElementCount(shape []int64) int64 {
+	if len(shape) == 0 {
+		return 1
+	}
+	count := int64(1)
+	for _, d := range shape {
+		if d < 0 {
+			return -1
+		}
+		count *= d
+	}
+	return count
+}


+	var bufPtr unsafe.Pointer
+	if totalLen > 0 {
+		bufPtr = unsafe.Pointer(&buf[0])
+	} else {
+		bufPtr = unsafe.Pointer(&buf)
+	}


+func (b *IOBinding) BindInput(name string, value *Tensor) error {
+	cName := C.CString(name)
+	defer C.free(unsafe.Pointer(cName))
+	return wrapErr("bind input", checkStatus(C.ort_BindInput(b.handle, cName, value.value)))
+}
+
+func (b *IOBinding) BindOutput(name string, value *Tensor) error {
+	cName := C.CString(name)
+	defer C.free(unsafe.Pointer(cName))
+	return wrapErr("bind output", checkStatus(C.ort_BindOutput(b.handle, cName, value.value)))
+}
+
+func (b *IOBinding) BindOutputToDevice(name string, memInfo *MemoryInfo) error {
+	cName := C.CString(name)
+	defer C.free(unsafe.Pointer(cName))
+	return wrapErr("bind output to device", checkStatus(C.ort_BindOutputToDevice(b.handle, cName, memInfo.handle)))
+}


+static const char *ort_dlerror(void) {
+    return dlerror();
+}


+	fn := C.ort_dlsym(handle, sym)
+	if fn == nil {
+		errMsg := C.GoString(C.ort_dlerror())
+		return nil, fmt.Errorf("dlsym OrtGetApiBase: %s", errMsg)
+	}


+	proc, err := dll.FindProc("OrtGetApiBase")
+	if err != nil {
+		return nil, fmt.Errorf("FindProc OrtGetApiBase: %w", err)
+	}


+	expected := int(count) * es
+	if len(data) != expected {
+		return nil, fmt.Errorf("ort: create tensor: data length %d does not match expected %d bytes", len(data), expected)
+	}


dannyota mentioned this pull request Jul 8, 2026

Do we have official Golang support for ONNXRuntime? #9786

Open

[Go] Support older ORT versions with API version fallback

5688439

Try API version 27 first, fall back to 17 for ORT 1.17-1.26. GetExecutionMode and IsMemPatternEnabled return a clear error on older libraries instead of crashing. Minimum ORT is now 1.17.

tianleiwu requested a review from Copilot July 11, 2026 16:53

Copilot started reviewing on behalf of tianleiwu July 11, 2026 16:54 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Go] Add Go bindings for ONNX Runtime C API#29615

[Go] Add Go bindings for ONNX Runtime C API#29615
dannyota wants to merge 4 commits into
microsoft:mainfrom
dannyota:dannyota/go-bindings

dannyota commented Jul 8, 2026

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dannyota commented Jul 8, 2026

Description

Motivation

What's included

Design

Tests

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

dannyota commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants